Data

The following analysis is built upon the International Energy Agency’s (IEA) IEA Headline Energy Data, which includes product data for 14 energy balance flows in kilotons of oil equivalent (ktoe) for 9 energy products over the years 1971-2014.

The IEA is an independent agency that works closely with the Organisation for Economic Co-operation and Development (OECD), an intergovernmental economic organisation with 35 member countries, founded in 1961 to stimulate economic progress and world trade (Wikipedia “OECD” 11/15/16).

This dataset can be downloaded from the IEA’s website: https://www.iea.org/statistics/

Overview

head(df,2)
##     Country                  Product              Flow X1971 X1972 X1973
## 1 Australia Coal, peat and oil shale Production (ktoe) 32680 39372 40251
## 2 Australia Coal, peat and oil shale    Imports (ktoe)     0     0     0
##   X1974 X1975 X1976 X1977 X1978 X1979 X1980 X1981 X1982 X1983 X1984 X1985
## 1 39179 44767 44161 48910 47947 51970 51898 62618 63335 72458 78958 83939
## 2     0     0     0     0     7     9     1     0     1     0     0     0
##   X1986  X1987 X1988  X1989  X1990  X1991  X1992  X1993  X1994  X1995
## 1 90500 102763 90074 100214 106103 114763 119139 119983 118687 127823
## 2     0      0     0      0      0      0      0      0      0      0
##    X1996  X1997  X1998  X1999  X2000  X2001  X2002  X2003  X2004  X2005
## 1 129956 140676 152427 154539 164578 178433 183863 183860 189189 201579
## 2      0      0      0      0      0      0      0     35     10     14
##    X2006  X2007  X2008  X2009  X2010  X2011  X2012  X2013  X2014
## 1 203808 214979 215910 223333 246564 233641 245351 264517 285440
## 2     33     28      5      4     43    161     87     62    126
names(df)
##  [1] "Country" "Product" "Flow"    "X1971"   "X1972"   "X1973"   "X1974"  
##  [8] "X1975"   "X1976"   "X1977"   "X1978"   "X1979"   "X1980"   "X1981"  
## [15] "X1982"   "X1983"   "X1984"   "X1985"   "X1986"   "X1987"   "X1988"  
## [22] "X1989"   "X1990"   "X1991"   "X1992"   "X1993"   "X1994"   "X1995"  
## [29] "X1996"   "X1997"   "X1998"   "X1999"   "X2000"   "X2001"   "X2002"  
## [36] "X2003"   "X2004"   "X2005"   "X2006"   "X2007"   "X2008"   "X2009"  
## [43] "X2010"   "X2011"   "X2012"   "X2013"   "X2014"
unique(df$Country)
##  [1] Australia                   Austria                    
##  [3] Belgium                     Canada                     
##  [5] Chile                       Czech Republic             
##  [7] Denmark                     Estonia                    
##  [9] Finland                     France                     
## [11] Germany                     Greece                     
## [13] Hungary                     Iceland                    
## [15] Ireland                     Israel                     
## [17] Italy                       Japan                      
## [19] Korea                       Luxembourg                 
## [21] Mexico                      Netherlands                
## [23] New Zealand                 Norway                     
## [25] Poland                      Portugal                   
## [27] Slovak Republic             Slovenia                   
## [29] Spain                       Sweden                     
## [31] Switzerland                 Turkey                     
## [33] United Kingdom              United States              
## [35] OECD Total                  People's Republic of China 
## [37] Indonesia                   Thailand                   
## [39] Non-OECD Total              Africa                     
## [41] Non-OECD Americas           Asia (including China)     
## [43] Non-OECD Europe and Eurasia Middle East                
## [45] World                      
## 45 Levels: Africa Asia (including China) Australia Austria ... World
unique(df$Product)
##  [1] Coal, peat and oil shale Crude,NGL and feedstocks
##  [3] Oil products             Natural gas             
##  [5] Nuclear                  Renewables and waste    
##  [7] Electricity              Heat                    
##  [9] Total                    Fossil Fuels            
## [11] Renewable sources       
## 11 Levels: Coal, peat and oil shale ... Total
unique(df$Flow)
##  [1] Production (ktoe)                                
##  [2] Imports (ktoe)                                   
##  [3] Exports (ktoe)                                   
##  [4] Total primary energy supply (ktoe)               
##  [5] Electricity plants (ktoe)                        
##  [6] CHP plants (ktoe)                                
##  [7] Heat plants (ktoe)                               
##  [8] Oil refineries, transformation (ktoe)            
##  [9] Total final consumption (ktoe)                   
## [10] Industry (ktoe)                                  
## [11] Transport (ktoe)                                 
## [12] Other (ktoe)                                     
## [13] of which: Residential (ktoe)                     
## [14] of which: Commerce and public services (ktoe)    
## [15] Electricity output (GWh)                         
## [16] Total CO2 emissions - Fuel Combustion (Mt of CO2)
## 16 Levels: CHP plants (ktoe) ... Transport (ktoe)

This datset contains 52 variables with 5,895 observations, with annual data encompassing 1971 to 2014 (5 variables were excluded because they were empty; these were projected placeholder variables for 2015 total and quarterly data). The data are condusive for within-country comparisons between energy products and between-country comparisions of specific energy products. In addition to country data, world totals and regional data are also provided.

To capture some of the breadth of these data, I have organized this report by analyses pertainting to USA, World and Regions.

USA

The dataset contains 16 levels of energy flow categories, and I chose to focus my analysis on imports and exports. To get a snapshot of US energy imports, I plotted a pie chart of 2014 product percentages. To do so, I excluded two products that had no data (Heat and Nuclear), and combined the three smallest imported products (Crude, Renewables, and Electricity) for greater legilibility.

Looking just at this 2014 snapshot, Crude makes up a whopping 75% of US energy imports. The supporting documentation for this dataset defined crude as crude oil, natural gas liquids, refinery feedstocks, and hydrocarbons. I wanted to see if these energy product proportions persisted across the entire timeframe of the dataset, so I plotted the longitduinal data of each energy product import together.

Crude imports surpass the next leading energy import by a large margin, and seem worthy of further investigation. Before analyzing them further, I wanted to see the dynamics of the other energy products in a plot where crude was removed so as not to skew the magnitude of the scale.

This helped me see some interesting patterns in gas (green) and oil (purple), particularly their mutual rise and fall during the second half of the timeframe. I isolated these two products to look more closely at their interaction.

I noted the drop off of imports around 2008, and would be interested to investigate further to see if this could be recession related (perhaps by comparing other OECD countries during this time period).

US oil imports and exports

In order to continue moving through my dataset, I switched my attention to comparing US imports with exports. When it comes to energy products, imports only tell half of the story, and I was curious to see if the US showed net deficits or net gains.

I chose to focus my attention on oil.Oil in this context includes refinery gas, ethane, aviation gasoline, motor gasoline, jet fuels, kerosene, gas/disel oil, fuel oil, and other oil products.

I could immediately see two distinct periods of time: 1) when imports were greater than exports, and 2) when exports were greater than imports. I shaded these in to see them more clearly.

I then calculated and plotted the total (ktoe) of the observed net gain.

I was intrigued by the parallel rise of imports and exports between around 1995 to 2006, so I calculated and plotted their correlation for this period.

export<- subset(
  df.imports_exports_USA$Exports..ktoe., 
  df.imports_exports_USA$Year < as.Date('2006', '%Y') & 
    df.imports_exports_USA$Year > as.Date('1995', '%Y')
  )

import <- subset(
  df.imports_exports_USA$Imports..ktoe., 
  df.imports_exports_USA$Year < as.Date('2006', '%Y') & 
    df.imports_exports_USA$Year > as.Date('1995', '%Y')
  )

export <- abs(export)

lm(export~import)
## 
## Call:
## lm(formula = export ~ import)
## 
## Coefficients:
## (Intercept)       import  
##   2.745e+04    2.188e-01

Their low correlation coeffecient led me to believe that one could not be used as a suitable predictor of the other for this period.

US crude

I shifted my attention back to crude, and plotted US crude imports against US crude exports.

The scale of the US crude net deficit was quite suprising, with exports only scarecly registering in the late 1990s and after around 2008. Also, as viewed initially in my import comparison, the magnitude of crude imports was far greater than what I had observed in oil. To help me visualize these relative magnitudes I plotted my US oil imports and exports against a backrdop of US crude imports and exports.

World

My US analysis led me to wonder how US energy compared to world totals. I started by plotting world energy imports.

World oil

Oil led the pack as the leading world energy import, so I singled it for closer examination.

World oil imports see a steady rise over time. I wanted to see if this was also true for exports, so I plotted them side by side.

export <- df.imports_exports_world$Exports..ktoe.
import <- df.imports_exports_world$Imports..ktoe.
lm(import~export)
## 
## Call:
## lm(formula = import ~ export)
## 
## Coefficients:
## (Intercept)       export  
##  -3.291e+04   -9.631e-01

Unlike the import-export disarpity I observed in US crude, world oil imports and exports seem to advance in lockstep. Their correlation coefficient was incredibly strong (-0.96).

I was curiou about the total net gain acculmulated over time, so I calculated and plotted that directly.

netgain <- sum(abs(df.imports_exports_world$Exports..ktoe.) - 
                 df.imports_exports_world$Imports..ktoe.)

netgain
## [1] 2586108

US vs. World (oil imports)

With over 2.5 Million ktoe in oil net gains, world totals paint a much fuller picture of the total oil market. To help me visualize this scale, I plotted US oil imports next to World oil imports.

Regions

In addition to data on specific OECD countries, this dataset includes regional data from non-OECD nations.

For comparison, I plotted OECD imports and exports alongside the non-OECD regions.

Regional percentages of total oil net gain

From these barcharts it was difficult to determine which country experienced the greatest oil net gain, so I calculated and plotted this directly as a pie chart.

Plotted in the pie cart below are the percentages of total positive regional net flow for oil from 1971-2014. These were calculated by subtracting crude imports from crude exports for each of the listed regions. The percentage listed for each region is that region’s total crude net flow for all years divided by the total positive crude net gain for all regions for all years (their slice of the regional net crude gain pie).

I removed Asia and OECD from the pie chart, as these two regions showed negative net oil flow.

As I expected, the Middle East is in the lead with 44% of all net positive oil flow across regions. I was curious to see if this was also true for crude.

For crude, the Middle East leads by an even bigger margin, with Africa, the second greatest percentage, coming in at a third of the Middle East’s percentage. Once again, Asia and OECD were ommitted from this barchart, as each had negative net crude flow.

US vs. OECD (crude flow)

Given the negative net crude for US and OECD, I wanted to see the percentage of negative OECD crude flow that could be accounted for by the US crude net deficit.

# Create OECD dataframe
df.ie_crude_OECD <- subset(df.ie_crude, Country == 'OECD Total')

# Plot OECD crude flow
ggplot(df.ie_crude_OECD, aes(x = Year)) +
  geom_line(aes(y = abs(Exports..ktoe.), color = 'Export')) + 
  geom_line(aes(y = Imports..ktoe., color = 'Import')) +  
   ylab('OECD crude (ktoe)') +
  geom_ribbon(
    data=subset(df.ie_crude_OECD, Year > as.Date('1970', '%Y')), 
    aes(x = Year, ymin=abs(Exports..ktoe.),ymax=Imports..ktoe.), 
    fill="red", 
    alpha="0.5") +
   scale_y_continuous(limits = c(0, 1800000), breaks = pretty_breaks(10)) +
  annotate(
    "text", 
    x = as.Date('2004', '%Y'), 
    y = 1000000, 
    label = "Net deficit")

# Calculate US percentage of OECD crude net loss
US_crude <- sum(df.ie_crude_USA$Imports..ktoe. - 
                  df.ie_crude_USA$Exports..ktoe.)
OECD_crude <- sum(df.ie_crude_OECD$Imports..ktoe. - 
                    df.ie_crude_OECD$Exports..ktoe.)
US_crude/OECD_crude
## [1] 0.2412082

While there are 34 countries in the OECD, the total US crude net deficit makes up 24% percent of the OECD total crude net deficit.

Three final charts

I began my analysis by examining US energy products, and quickly shifted my attention to oil. By plotting oil imports and exports together I was able to see that there was a net deficit between 1971 and 2009, which then shifted to a net gain as exports began to exceed imports. This trend increases over the last few years of the dataset.

Having seen how large US crude imports were in my initial analysis, I plotted these against exports and included a shaded depiction of the US crude net deificit. For comparison, I plotted this against a backdrop of the same relationship (crude imports vs. exports) for the entire OECD, and calculated the US percentage of this total crude net deficit (24%).

The IEA, the agency responsible for this datasest, is an idependent agency that works in close partnership with the OECD. Given this, it should come as no surprise that this dataset has extensive energy flow data for OECD partners, yet only overview regional data for non-OECD nations and regions.

That being said, even if we look just at regional crude oil net gains, we can begin to see outlines of the energy flow landscape of nations who export more crude than then import.

## [1]  11089208  -7214791  32822605   2496360   5946392 -47322953

Reflections

This analysis began with an overview of US energy imports, shifted to a deeper look at US oil and crude, drew comparisons to world crude flow, and ended with a look at regional net flow. This trajectory was inspired by initial observations in my first plot that showed oil and crude to be leading US imports. This analysis was also motivated by my desire to have a fuller picture of how imports related to exports over time, as such an analysis begins to paint a part of the US energy economic picture for these products.

While there were complete sets of datapoints for oil and crude, I was dissapointed to find little to know data for some of the other US products, particularly for Renewables. It was not clear from the documentation whether this lack of data is due to low US participaion in the renewables market or a biproduct of the data collection process. A deeper study of the IEA’s data collection methods would be required to answer this question, and would be advised before providing any analysis of these energy products with sparse or no data.

While I enjoyed the ktoe accuracy in these data, I found myself wishing to plot net deficits and gains in terms of costs (perhaps in USD or GBP). This would require cross-referencing energy trading prices at the time of data collection. Such analysis would help provide a better understanding of specific economic realities of these product types. This includes industry applications, trade restrictions, and efficiency as a fuel source. Such information, combined with cost, could help transform the data here into a more unified picture of global energy value distribution.